Search results for "multimodal interaction"
showing 10 items of 19 documents
“Do you understand (me)?” negotiating mutual understanding by using gaze and environmentally coupled gestures between two deaf signing participants
2020
Abstract In this paper we explore the use of multimodal and multilingual semiotic resources in interactions between two deaf signing participants, a researcher and an asylum seeker. The focus is on the use of gaze and environmentally coupled gestures. Drawing on multimodal analysis and linguistic ethnography, we demonstrate how gaze and environmentally coupled gestures are effective semiotic resources for reaching mutual understanding. The study provides insight into the challenges and opportunities (deaf) asylum seekers, researchers, and employees of reception centres or the state may encounter because of the asymmetrical language competencies. Our concern is that such asymmetrical situati…
Children’s Engagement with Mathematics in Kindergarten Mediated by the Use of Digital Tools
2013
The purpose of this chapter is to study children’s engagement with mathematics in kindergarten mediated by digital tools within the context of interactive whiteboard (IWB). Our study reports from a research project aiming at analysing in what ways digital tools may nurture children’s appropriation processes relative to mathematics. In adopting a sociocultural perspective on learning and development, observations together with video recordings have been analysed. Our study shows that the children make sense of the digital tools and are able to apply the tools purposefully due to the interaction with an adult within the zone of proximal development (ZPD). The children externalise their reason…
Bodily Engagement in Multimodal Interaction
2010
The creative processes of interaction design operate in terms we generally use for conceptualising human-computer interaction (HCI). Therefore the prevailing design paradigm provides a framework that essentially affects and guides the design process. We argue that the current mainstream design paradigm for multimodal user-interfaces takes human sensory-motor modalities and the related userinterface technologies as separate channels of communication between user and an application. Within such a conceptualisation, multimodality implies the use of different technical devices in interaction design. This chapter outlines an alternative design paradigm, which is based on an action-oriented persp…
DIGITALE MODERATION IN DER DAF-LEHRENDENBILDUNG
2022
The aim of this paper is to show patterns in the moderation of video conferences. The corpus used comes from the Erasmus+ project LEELU (development of teaching skills for extensive reading lessons). The subject of the investigation was how changes in the agenda are negotiated at the beginning of the meetings. A multimodal method based on conversation analysis was chosen for the analysis.
Focal social actions through which space is configured and reconfigured when orienting to a Finnish Sign Language class
2018
Abstract This paper focuses on how signing students organise themselves spatially in social interactions in a university lecture hall. One may view space as a concrete location, a social construct, and a normative actor with historical trajectories. The study addresses the question, ‘What are the mediated actions through which the students and teacher (re)configure space for participating in a class?' Following a methodological framework of Mediated Discourse Analysis and multimodal interaction analysis, I approach this question by examining the social actions occurring when entering a lecture hall. The primary data includes video recordings, photos, and participatory observations, document…
Encapsulació lèxica i avaluació en el debat parlamentari
2018
Lexical encapsulation consists of a series of abstract unspecific nouns (fact, plan...) referring to predicative antecedents. This study is based on a corpus of parliamentary debate in English, Catalan and Spanish (PD) and deals with encapsulation as a complex lexical cohesion device which allows the addressor to evaluate the information of the debates in various ways. Our work tries to highlight the role of lexical encapsulation to reflect the addressor’s positioning with respect to the topic discussed, and to establish whether there are outstanding cross-linguistic differences. In order to delimit the diverse evaluative strategies, an approach from the perspective of prototype theory is a…
Interweaving visual and audio-haptic augmented reality for urban exploration
2018
While ordinary touchscreen-based interfaces on urban explorer applications draw much of a user's attention onto the screen, visual and audio-haptic augmented reality interfaces have emerged as the two main streams for enabling direct focus on the surroundings. However, neither interface alone satisfies users in the highly dynamic urban environment. This research investigates how the two complementary augmentation can coexist on one system and how people adapt to the situation by selecting the more suitable interface. A prototype was deployed in a field experiment in which participants explored points of interest in an urban environment with both interfaces. The engagement with the surroundi…
A wearable multimodal interface for exploring urban points of interest
2015
Locating points of interest (POIs) in cities is typically facilitated by visual aids such as paper maps, brochures, and mobile applications. However, these techniques require visual attention, which ideally should be on the surroundings. Non-visual techniques for navigating towards specific POIs typically lack support for free exploration of the city or more detailed guidance. To overcome these issues, we propose a multimodal, wearable system for alerting the user of nearby recommended POIs. The system, built around a tactile glove, provides audio-tactile cues when a new POI is in the vicinity, and more detailed information and guidance if the user expresses interest in this POI. We evaluat…
Calm Down – Exploiting Sensorimotor Entrainment in Breathing Regulation Application
2011
Various phenomena in human life are related to different kinds of rhythms. Not only are our bodily functions based on rhythms, but also much of the interaction with our environment is related to them. In this study, we explore breathing regulation and how it could be supported with an interactive application. The application is based on the concept of entrainment, in which two interacting entities adjust to a common rhythm. The focus is in the design of interaction elements which support entrainment process. A user study of a prototype application is also reported in the paper. The results indicate that the approach is promising and has potential in opening new perspectives to humancomputer…
Combining intelligent recommendation and mixed reality in itineraries for urban exploration
2017
Exploration of points of interest (POI) in urban environments is challenging for the large amount of items near or reachable by the user and for the modality hindrances due to reduced manual flexibility and competing visual attention. We propose to combine different modalities, VR, AR, haptics-audio interfaces, with intelligent recommendation based on a computational method combining different data graph overlays: social, personal and search-time user input. We integrate such features in flexible itineraries that aid different phases and aspects of exploration.